This invention relates generally to computer memory systems and more particularly to an access method for reading from a shared memory to an I/O device.
In a computer system, problems often arise when more than one device attempts to access information stored in a memory location. While multiple devices can access a single memory location, if one of the accessing devices attempts to update the information in the memory location, without informing the other devices who also have access to the specific memory location, data mismatches may occur resulting in a loss of data coherency.
To speed access and minimize data latency, some memory accessing devices use a local memory cache. Within the local cache the device may store a copy of information which it has recently accessed. Thus, there may be several copies of information scattered throughout a system.
When a system implements local cache, main and cache memory may be organized into cache lines. A cache line is typically 64 bytes of data. Therefore, when a device attempts to access a specific memory location the cache controller first searches its local cache to determine if it already has a copy of the information available. If a copy of the requested memory location is not currently stored in the local cache, the cache controller attempts to obtain a copy of the cache line from the system memory controller. If the information is available in the local cache, the device will use the information in the local cache. Issues arise when multiple devices attempt to access the same cache line, and each device stores copies of this information in local cache. Not only must access conflicts be resolved, but procedures must be implemented to ensure coherency of the various copies of the data contained in the multiple caches and in the main memory.
Numerous protocols exist which maintain cache coherency across multiple caches and main memory. One such protocol is called MESI. MESI protocol, which is described in detail in M. Papamarcos and J. Patel, xe2x80x9cA Low Overhead Coherent Solution for Multiprocessors with Private Cache Memories,xe2x80x9d in Proceedings of the 11th International Symposium on Computer Architecture, IEEE, New York (1984), pp. 348-354, incorporated herein by reference. MESI stands for Modified, Exclusive, Shared, Invalid. Under the MESI protocol, a cache line is categorized according to its use. A modified cache line indicates that the particular line has been written to by the cache that is the current owner of the line. An exclusive cache line indicates that a cache has exclusive ownership of the cache line, which will allow the cache controller to modify the cache line. A shared cache line indicates that one, or more than one, cache(s) have ownership of the line. A shared cache line is considered read only and any device under the cache may read the line but no one is allowed to write to the cache. A cache line with no owner identifies a cache line whose data may not be valid since the cache no longer owns the cache line.
While MESI is a standard term in the industry, other classifications or nomenclature are frequency employed. Modified cache lines are typically referred to as private dirty. Exclusive cache lines are typically referred to as private cache lines. Private cache lines which have not been updated are typically referred to as private clean cache lines.
If a specific device requires access to a specific memory location it will check its local cache to determine if the information is available there. If the information is not currently contained within the local cache, the cache controller will go to main memory to access the information. Before requesting the cache line from the system memory controller, the cache controller decides what type of ownership for the line it will seek (i.e. shared, private, etc.). If a cache wants private ownership of a cache line, the system memory controller will ensure that no cache has the same line. The system memory controller will not allow any other cache to get ownership of this line so the cache""s access to the line will be private. If the cache""s access to the line is shared, more than one cache may have the same line as shared simultaneously.
A cache must have private or exclusive ownership of a cache line to modify the line. That is, other copies may not be relied upon as being valid until after the data is modified and an updated version is supplied. If the cache line is currently categorized as read only, the cache which needs to update the information must make a request to the system memory controller, for exclusive access to the cache line. The system memory controller then identifies any other caches having access to the cache line, and makes the necessary arrangements for the other caches to relinquish access and for the requesting cache to have exclusive use of the cache line.
One method for a cache to obtain the exclusive use of a cache line is for the memory controller to invalidate other copies of the cache line currently in use. Once other caches"" access to the cache line has been invalidated, the remaining cache has exclusive use of the cache line and can update the cache line accordingly.
One of the methods to implement cache coherency is a directory-based system where rather than sending each transaction to every other agent or other processor in the system a table is maintained for each cache line which indicates which agent(s) have the cache line. The system memory controller consults with the directory to see the status of the cache line before it allows a cache to get data. For example, another cache has the cache line as private, the memory controller recalls the cache line from the original owner.
A device first checks its local cache to determine whether the information is available. If the information it requires is not locally available, the cache controller servicing the device sends a read request for shared ownership of the cache line via the system interconnect fabric to the system memory controller. If the requested cache line is shared or is not in use by any other cache, the system memory controller sends the data from main memory to the requesting cache directory reflecting the new owner of the cache line in shared mode. Once the requesting device has completed its use of the cache line, the cache controller sends a message relinquishing ownership of the cache line to the system memory controller. Upon receipt of his message the system memory controller removes the cache as a shared owner from its directory tag. If the cache controller had requested private access to the cache line but had updated the information stored within the cache line, the cache controller also sends updated information to the system memory controller. Each of these transactions between the cache controller and the system memory controller consume a portion of the interconnect fabric bandwidth. Additionally the system memory controller bandwidth is also affected by these transactions. A reduction in the number of steps required for the cache controller to acquire and relinquish ownership of the cache line would provide a corresponding improvement to overall system performance.
Data requested and accessed by the cache controller can be one of several types. First, the cache controller request and memory access can be associated with payload data. Payload data consists of a large data transfer so that it is often handled as blocks of 1 to 4 KB of data. A second type of data that can be requested and accessed by the cache controller is control data and is generally smaller in size. This information is typically between 4 and 32 KB and can be accommodated in one cache line for most applications.
In addition to the shared and private memory requests discussed previously, a third type of access to information stored in system memory exists. A xe2x80x9cread currentxe2x80x9d transaction results in the requesting cache controllers receiving coherent data of the cache line at the time of the request but does not affect the ownership of the cache line. A read current transaction guarantees that the information contained within the cache line was up to date at the time the data was obtained from the system memory controller. One of the advantages of a read current request is that, after the device has used the data, the cache controller does not have to relinquish ownership of the cache line since the read current had no effect on the ownership of the cache line within the system. The elimination of this third step in the normal cache controller access of memory location improves the useable bandwidth in the system fabric.
Contentions can arise between two caches which both attempt to obtain ownership of a cache line. Effectively, after the system memory controller has assigned one cache ownership of the cache line, the second cache can request ownership (prior to the first cache""s use of the cache line), resulting in the first cache losing ownership of the cache line. Similarly, after the system memory controller assigns ownership of the cache line to the second cache, but before the second cache has used the cache line, the first cache can again request the cache line. In some cases this contention between the requesting caches can result in the cache line xe2x80x9cping-pongingxe2x80x9d between the two contending caches. This prevents either device under the two caches from making forward progress. It also results in increased latency for the devices/processors accessing the cache line.
A need currently exists to improve the useable bandwidth for large payload transfers by eliminating one of the three steps in a normal data transfer. Additionally, a further need exists to eliminate the presence of xe2x80x9cping-pongingxe2x80x9d which may occur between contending caches.
These and other objects, features and technical advantages are achieved by a system and method which according to an aspect of the invention, a shared memory providing access to a plurality of agents (e.g., processor, cells of processors, I/O controllers, etc.) includes a memory and a memory controller. The memory controller selectively provides memory access to the agents in both coherent and read current modes of operation. In the coherent mode, the memory controller ensures that an agent""s cache reflects the most up to date data. Using, for example, a MESI type protocol, the memory controller limits access to memory so that only one xe2x80x9cownerxe2x80x9d gets the cache line at any time for write access and that, during the extension of these write privileges, no other agent has a valid copy of the data subject to being updated. Thus, the memory controller implements a first set of rule in the coherent mode of operation to insure that all copies of data stored by the agents are coherent with data stored in the memory. In a read current mode of access, a read-once segment of data is copied to an agent with the agent implementing a second set of rules to eliminate the possibility that the data might become stale prior to use or that it be misused by another agent or process. Thus, in the read current, an xe2x80x9cuncontrolledxe2x80x9d copy of the data is released subject to certain restrictions to be implemented by the receiving agent as defined by a second set of rules.
According to a feature of the invention, the second set of rules require that the agent""s copy of data be provided as an output and then invalidated after use. These rules further require that the uncontrolled data, i.e., the agent""s copy, be limited in size and terminate in a predetermined block boundary. These limitations restrict the read current data from causing data corruption including the scenario where the requesting device were to experience a failure such that protection of the data might be compromised.
According to another feature of the invention, the second set of rules require that the copy of data be accessed by an external device in a sequentially incremental fashion and that, upon completion of the transfer, the copy of data be flushed from the host agent""s cache. Again, these rules are directed at ensuring that the data is used immediately and not permitted to become stale, and that the data is flushed, i.e., is read once and only once. Another rule implemented by the agents according to the read current protocol is a requirement that detection of an error condition causes the copy of data to be flushed. These errors include, for example, a failure of the requesting agent such that it cannot comply with or implement one or more of the safeguard required to protect this uncontrolled data from misuse.
According to another feature of the invention, different sets of rules, of subsets of rules, are used and applied to provide payload versus control data and traffic. The dichotomy accommodates differences between these data types, include typical sizes and uses of the data.
According to another aspect of the invention, a shared memory providing access to a plurality of agents includes both a memory and a memory controller. The memory controller selectively provides memory access to the agents in both coherent and read current modes of operation. In the coherent mode of operation, the memory controller follows cache coherence protocol in prior art such as the MESI protocol. In the read current mode of operation, the memory controller provides memory access to an agent so that a copy of data stored in the memory is retrieved subject to the agent implementing a set of rules to avoid the data going stale or being misused. The agents may implement theses rules using appropriate control logic.
The read current rules implemented by the agents include:
(i) data supplied in the read current mode of operation is to be sequentially and incrementally transferred by the receiving agent beginning at a starting address through an end of a predetermined block address;
(ii) data supplied in the read current mode of operation is to be flushed by the receiving agent upon completion of a transfer operation;
(iii) data supplied in the read current mode of operation is to be flushed by one of the agents upon detection of a first fault condition;
(iv) data supplied in the read current mode of operation is to be flushed upon an interruption of a transfer of the data;
(v) fetching operations to supply the copy of data are to be terminated upon detection of a second fault condition; and
(vi) in response to a third fault condition, fetching operations to supply the copy of data are to be completed through an end of a corresponding DMA block address and the copy of the data is to be flushed.
According to another aspect of the invention, a method of providing memory access to a plurality of agents selectively provides memory access in a coherent and read current mode of operation. A first set of rules, initiated by the controller, ensures that all copies of data stored by the agents are coherent with data stored in the memory. A second set of rules supporting read current memory access are implemented by the agents without controller monitoring and/or intervention so as to provide the agents with read only access to data stored in the memory. Read current procedures implemented by a participating agent include creating a cache copy of data using the read current memory access, transferring to a peripheral device the cache copy of data, and then flushing the cache copy of data.
The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims.